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effect In this study is probably due to the difference in CMI and CAI , i.e. 

CMI does not present instructional material that may require some fo>'m of 
memory aid for effective learning- 

There was a significant difference in the performance of the CRT and teletype 
groups on the course product. A further analysis of the data revealed that as 
a whole the CRT completed the units on the concepts at an earlier data than 
did the teletype group as well as spending less t^e interacting with the 
system. Apparently the students in the CRT group began work on their products 
earlier than the teletype group. 

As was expected the CRT group spent significantly less time signed-on to the 
CMI system. This probably »^eflects the operating speeds of the terminal devices 
rather than any other factor. 
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AN INVESTIGATION OF THE EFFECTS OF TWO 
TvpES OF INSTRUCTIONAL TERMINALS IN 
COMPUTER-MANAGED INSTRUCTION 

Bobby R- Brown, Wallace H. Hannum, and Walte*^ Dick 

Abstract 

This paper reports the findings of an experiment in which 
28 students taking a c'^edit earning graduate level course in 
techniques of programmed instruction were randomly assigned to either 
cathode ray tube or teletype terminals. 

Results from the analysis of the final concept test data revealed 
that students performed equally well regardless of terminal device 
Apparently the information load in the CMI system is sufficiently '’ow to allow 
acceptable performance without the necessity for some form of n-errory support. 
The difference in error rate is interpreted to reveal a diminished ef'^ect 
of memory support on reducing errors. This effect 0+ memory support has 
been found in CRT's using CAI learning materials. The failure to find this 
effect in this study is probably due to the difference in CMI and CAI 
i.e., CMI does not present instructional material that may require some 
form of memory aid for effective learning. 

There was a significant difference in the performance of the 
CRT and teletype groups on the course product, A further analysis of the 
data revealed that as a whole the CRT completed the units on the concepts 
at an earlier date than did the teletype group as well as spending 
less time interacting with the system. Apparently the students in the 
CRT group began work on their products earlier than the teletype group. 



As was expected the CRT group spent significantly less 
time signed-on to the CMI system. This probably reflects the 
operating speeds of the terminal devices rather than any other 



factor. 



AN INVESTIGATION OF THE EFFECTS OF TWO 



TYPES OF INSTRUCTIONAL TERMINALS IN 
COMPUTER-MANAGED INSTRUCTION 

Bobby R, Brown, Wallace H. Hannum, and Walter Dick 

One of the most often suggested ways to improve education is 
that of ind'i vidua II i zing the Instructional process to more adequately 
meet the needs of each student. The problem of individualizing 
instruction has intrigued educators for several decades. Early 
attempts at individualization were reported by l'<ashburne (1926), and 
Parkhurst (1922). Although these plans were well formulated, progress 
in the implementation of individualized instructional programs has been 
meager. 

This lack of progress is, in part, a function of administrative 
problems associated with individualized instructional programs. In a 
truly individualized progra(n, all students in a class would pursue dif- 
ferent instructional sequences, at varying rates, which places a manage- 
ment burden upon the teacher. Recording the progress of each student, 
and prescribing instruction for him is a task which can require a large 
amount of time from several persons. The work with individually pre- 
scribed instruction (IPI) done at Oakleaf School in conjunction with the 
University of Pittsburgh demonstrated the magnitude of the problem of 
non-automated information management (Cooley and Glaser, 1969). Initially, 
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the IPI program used several clerica? assistants to a?d teachers in the 
handling of the information necessary for the management of such an 
instructional system. 

There have been several recent attempts at applying computer 
technology to the managerial problem underlying individua.Mzed instruct'^on 
programs (Coulson, 1968; Schuer, 1967; Flanagan, 1970; Hagerty, 1970). 
Althouah these studies differ in a variety of ways, they are all suffic- 
iently similar to be labeled computer-managed instruction (CMI) projects 
(Morgan, 1969). 

In CMI, the computer can function as evaluator, diagnostician, 
and prescriber of instruction as well as a recorder of student progress. 
Rather than serving as our instructional presentation service as in 
computer-assisted instruction (CAI), the computer monitors the student's 
learning through a step-by-step evaluation of his progress. Thus in CMI, 
the actual learning does not occur at the computer terminal, but rather 
"off-line" in a more conventional (but usually individual Ued) fashion. 
Since on-line instruction is not a feature of CMI, the computer costs of 
this approach are much lower than CAI. Hagerty 11970) reports a com- 
parison of the costs of instruction between CMI and traditional Instruc- 
tion at the University level. She found that CMI costs were approximately 
one-half to one-third the cost of conventional graduate instruction. 

There are two basic types of CMI systems: batched processing and 

terminal oriented. There are a number of projects which have utilized a 
batched processing system in their operations such as Flanagan's Project 
Plan (1970), Coul son's work at Systems Development Corporation (1968), and 
O'Dierno's (1969) work at the New York Institute of Technology. In these 




9 



3 






projects, students are directed to learning matet^ia-'s based on progress 
information supplied by the computer to their teacher. Students' instruc- 
tion and testing is all performed with conventiona? paper and pencil pro- 
cedures. Test answer sheets are read by optical scanners and the data 
is transferred to computers for analysis and reporting, In turn, the 
reports are supplied to the instructor and/or the student- 

A terminal -oriented CMI system has been developed at the Florida 
State University CAI Center. The significant feature of this interactive 
CMI approach is that the diagnostic evaluations and learning prescriptions 
take place in real-time via an interaction between the student and the 
computer system. This system has the virtue of providing immediate correc- 
tive feedback as opposed to the usual 1-2 day wait In the batched systems. 

The initial implementation of this interactive system has been 
reported by Hagerty (1970) and Gallagher (1970). The ps^esent study extends 
the investigation of an interactive CMI system by focusing on the use of 
two different terminal devices for the student-machine interface. The 
terminal devices used were cathode-ray tubes (CRT) and teletypes (TTY). 
Information can be displayed at a faster rate on the TV-like screen of the 
CRT as compared to the typing rate of the teletypes, but the CRT's do not 
provide a hardcopy of this information for future reference. Due to the 
presentation rate, the CRT's may be more efficient terminal devices than 
teletypes for use in an interactive CMI system. However, the teletypes 
generate a printout of the student's interaction with the system which can 
be used for future study and review. Thus, the teletypes might be con- 
sidered the more desirable terminal device in terms of fostering student 
learning and retention. 
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The major purpose of this study was to ’"nvesfigate the perfo'^mance 
of students in a CMI course when they used two types of terminal devices. 

The role of individual differences with regard to terminal device was 
also investigated. A study by Dick and Latta (1969) reported a signifi- 
cant ability by treatment interaction in a comparison of programmed 
instruction and CRT-oriented CAI instructlono They found that low ability 
junior high school students performed better on PI than CAI, whereas there 
were no differences for high ability students. They tendered the explan- 
ation that low ability students may be unable to cope with the information 
load from the CRT since they don't have a copy of previous information 
to assist them. In the current study, g'^aduate record exam (GRE) and 
gradepoint average (GPA) were used as ind'’ces of graduate student ability. 

STATEMENT OF THE PROBLEM 

This investigation examined the effect of two d'^fferent terminal 
devices (CRT and teletype) on both performance and t'me required to complete 
a graduate level course offered via an interactive computer -managed instruc- 
tion system. 

In investigating differences due to terminal device, several 
indices of student performance were used. These Include: (1) Student scores 
on a criterion-referenced test of the major objectives involved in the course; 
(2) Instructor rating of each student's project for the course which 
included the development of a unit of programmed instruction and accompanying 
documentation; and (3) Time spent at the CMI terminal. 

In addition to these three primary indices, serveral other measures 
were obtained. These include the error rate on performance on the computer 
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terminal, the number of objectives 'n which criterion performance was not 
initially reached, and the computer costs per student for participating in 
the course. 



SUBJECTS 

The subjects for this study consisted of 28 graduate students who 
enrolled for the course "Techniques of Programmed Instruction" in the 
Department of Educational Research at the Florida State University during 
the Spring Quarter, 1970. There were 4 females and 24 male subjects. 

The subjects were randomly assigned to either CRT or teletype terminals. 

APPARATUS 

An IBM 1500 Instructional System was used to conduct this study. 
The IBM 1500 Instructional Display terminal equipped with light pen and 
typewriter keyboard served as one terminal device. The other terminal 
device used was a teletype. The addition of a DEC 680 System under :he 
control of a PDP/8 Computer provides the capability to drive both CRT's 
and teletypes by the same system. 

MATERIALS AND PROGRAM DESCRIPTION 

The materials used in this study consisted of a course guide 
distributed to the students at the start of the course, and numerous 
articles and chapters in books that were referenced to specific course 
objectives. The course guide included a task analysis of the course 
(see Appendix A), and for each task the guide gave the overall goal, 
specific behavioral object! ve(s ) , a sample test item, and primary and 
secondary references. These references were reproduced and kept in the 
Center's library which was open daily for student use. For each behavioral 
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objective, approximately fifteen test items were written and entered into 
a test item pool in the CMI system. 

The CMI system was programmed in Coursewriter II, the CAI 
language which is available for the IBM 1500 CAI System. The •{'unctions 
of the CAI system consisted of record keep-ing, testing, diagnosis of 
weakness, and remediation. The student was not presented actual instruc- 
tional materials on the terminals. A flow diagram of this interactive 
system for one unit in the course in presented in Figure 1. 

This figure represents the path followed within each unit of 
the course. The first 12 tasks covered the concepts involved in the 
preparation of programmed instruction, the last 8 tasks involved the 
students' preparation of their own programmed unit and the documentation 
for it. The unit tests for each of the first 12 units consisted of 5 
multiple choice or true-false questions selected at random from the test 
item pool for each objective for the task., The criterion for these task 
tests was set at B0% or 4 our of 5 items for each objective. 

The tests for the productive port’ton of the course, tasks 13-20, 
consisted of a series of questions that the instructor would have asked 
the student about his product. It is possible to regard these tests as a 
simulated interview between student and instructor in which the instructor 
asks the student yes or no type questions about what he has done. 

PROCEDURES 

The treatment period for this investigation was ten weeks, the 
length of the Spring academic quarter at Florida State University. At the 
first meeting of the class, the procedures for the course were described. 
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The students were told how they would p/'ocede through the course via 
computer management and were given a copy of the course guide. During this 
class they were told that they wouio not meet again until the last day of 
the quarter when their product for the course was due. 

The students were allowed to complete the units of the course in 
any order they desired at any rate they wished, with the restriction that 
they must finish all the units and comp’'ete their product by the end of 
the quarter. When a student completed the first 12 units covering the 
concepts involved in programmed instruct on, be was given a paper and 
pencil examination designed to measure his knowledge of the concepts (see 
Appendix B). When the student had turnea in their programmed instruction 
documentation, a score was assigned to this product using an instructor 
rating sheet (see Appendix C). 

The class met briefly the last day of the quarter primarily to 
discuss their reactions to the course, any problems they encountered, and 
to provide some closure for the course, It should be noted that the class 
only met on the first and last days of the quarter . They were required 
to finish the first 12 tasks by midterm and to have the documentation 
finished by the end of the quarter. 

RESULTS 

The main focus of this study was the investigation of the effect 
of two different terminal devices on student performance in a course offered 
via an interactive CMI system. The primary indices of studnet performance 
were scores on the final test over the conceptual tasks, scores on the 
course project (development and documentation of PI text), and total time 
spent by each student at the computer terminal,, 
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Table 1 presents the means and sianda''d deviations for the two 
experimental groups on these three performance indices 



TABLE 1 

Performance in CMI Course by Students Using 
Different Termina) Dev'ices 



Groups 




Concept 

Test 


Product 
Eval uation 


Time (Min.) 


CRT 


Mean 


90,64 


52.00 


261.14 


N = - 


Standard Dev. 


7.15 


6,13 


329.43 


TT 


Mean 


89 ,86 


45,36 


353.78 


N = - 


Standard Dev, 


8 06 


5,87 


327.84 


TOTAL 


Mean 


90 25 


48.83 


307.46 


N = - 


Standard Dev, 


7 , 49 


6.78 


325.92 



It can be readily seen from the data in Table 1 that both groups 
performed equally well on the test o</er- the concepts Involved in the 
course. The CRT group had superioe scopes on the course project and spent 
less time actually working at a terminal o 

In order to evaluate the differences on the concept test, an 
analysis of covariance was performed using the biomedical general linear 
hypothesis program (Dixon, 1968). The results of this analysis using 
pretest scores for the covariant yielded a F-ratio of 1.52. The F-ratio 
obtained from this analysis is not significant at the .05 confidence level. 
This indicates that there were no differences in the posttest over the 
concepts when the scores were adjusted for pretest differences. 
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An one-way analysis of variance was performed on the scores on 
the course product to examine dif-f'erences between the two groups ^ This 
analysis is presented in Table 2. 



TABLE 2 





Analysis of Variance 


for Product 


Scores 




Source 


Sum of Squares 


of 


Mean Squares 


F 


Between 


252.7589 


1 


252.7589 


6.99 


Within 


758.5455 


2i 


36.1212 




Total 


1011.3043 


22 







The obtained F-ratio is significant at the 05 leveU The pro- 
ducts of the CRT group were rated as significantly superior to the 
products of the teletype group. 

In order to evaluate the differences between the two groups in 
the amount of time spent on the computer terminal a Mann-Whitney U statistic 
was calculated. This resulted in a U of 33, significant at the .02 level 
indicating that the CRT group spent significantly less time signed on to 
the computer during this study. 

In addition to these main analys>?s. several supplemental factors 
including the number of errors on the unit tests, the number of times 
criterion performance was not reached on the first attempt, and the costs 
associated with the use of the two terminals was investigated. Table 3 
shows the comparison between the two groups on these factors. 
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TABLE 3 



Mean Errors, Criterion Failures, and Associated Costs 





Mean 

Errors 


Mean Failure to 
Achieve Criterion 


Computer Costs 
per Student 


CRT 


37.21 


5.14 


$14.48 




13.23 


1.99 




TT 


33.71 


5.07 


$19.58 




19.08 


1.49 





The total number of task test errors was not significantly differ- 
ent for the two groups. There was also no difference in the number of 
tasks on which criterion performance was not reached on the first attempt. 

The analysis of the cost per student indicates that the teletype was the 
more expensive terminal device. These cost figures were computed by 
multiplying the number of student hours spent on the system by the figure 
that reflects the current cost for time at the FSU CAI Center. This 
figure is $3.33 per student per hour regardless of terminal device. If 
the actual costs of the terminal devices were included, the teletype 
groups' costs would be even lower. 

The general finding was that, regardless of terminal device, 
students perform equally well on the test on conceptual material. However, 
the CRT group spent less time interacting with the computer and also developed 
better course projects. 

In order to access the role of individual differences within this 
CMI system, several indices of individual differences were employed: (1) 

GRE total scores; (2) grade-point average (GPA); and (3) anxiety as measured 
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by the State Trait Anxiety Inventory (Sp-’elberger, Gorsuch, and Lushene, 
1968). The correlations between these variables and the performance 
variables are presented below in Table 4. 



TABLE 4 

Relation Between Performance Measures and 
Individual Difference Variables 





GRE 


GPA 


Anxiety 


Midterm Test 


r ii 


.20 


-.07 


Product grade 


.13 


.38 


.15 


Number of Failures to 
Achieve Criterion 


-,0i 


-,67* 


.38 



*p < .05 

The only significant coefficient in this table is the correlation 
between GPA and the number of times criterion performance was not initially 
reached in the course evaluations on the terminals. This correlation would 
indicate that persons with higher ability as measured by GPA tended to more 
often reach criterion performance on the first attempt. 

The GRE scores were not related to any of the performance measures 
in the course. The anxiety scale also was not significantly correlated with 
performance. There was no evidence of differential performance of students 
of differing abilities. 
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DISCUSSION 

Results from the analysis of the concept test reveal that students 
perform equally ivell regardless of terminal device. Apparently the 
information load in the CMI system is sufficiently low to allow acceptable 
performance without the necessity for some form of memory support. The 
difference in error rate is interpreted to reveal a diminished effect of 
memory support on reducing errors. This effect of memory support has been 
found in CRT's using CAI learning materials. The failure to find this 
effect in this study is probably due to the difference in CMI and CAI, 
i.e., CMI does not present instructional material that may require some 
form of memory aid for effective learning. 

There was a significant difference in the performance of the CRT 
and teletype groups on the course product. A further analysis of the 
data revealed that as a whole, the CRT completed the units on the concepts 
at an earlier date than did the teletype group as well as spending less 
time interacting with the system. Apparently, the students in the CRT 
group began work on their products earlier than the teletype group. This 
interpretation is of course only conjectual . 

As was expected the CRT group spent significantly less time signed 
on to the CMI system. This probably reflects the operating speeds of the 
terminal devices rather than any other factor. 
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; Describe the use 
' oX Systems Analysis 
:• in developing materials 
• for a medium other than 




( 




PI 


' Distinguish stra- 
tegies available 
. with PI 


Classify B.O. in terms i 
• 9 , of the tasks required j 
of the learner 





ly : identify types of 7 • Identify behavioral objectives ; 
PI frames . which have been written correctly •; 



Distinguish PI ;4 . List requirements J Identify entry behavior | 

and non -PI [ for task analysis ^ of students 



List requirements for choosing * 
an area of instruction 



Identity and describe comp onenSs 
of the Systems Approach 



Task Analysis of Techniques of Programmed Instruction, showing cognitive 
and productive units , 
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APPENDIX B 



MIDTERM EXAMINA^ON FOR PROGRAMMED INSTRUCTION COURSE 



’h'.' examination cove':- the thi/teen cognitive units of the programmed 
'•^st'^uction course. Read and answer all questions carefu11>\ 

1 Briefly explain the ma.jor use of the Systems Approach in Education. 

2 list, in order, the steps in Dick's or Hansen's Systems Approach Model - 

3- List the f'jree elements baste to all Systems Approach Models. 

4- List five of the ba; "i ; -equirements necessary in choosing a subject 
m-atter area to be prog^'ammed. 

5. Briefly define: 

a. Step size-- 

b. Explicit response -- 
c Feedback-- 

d. Prompt-- 

6 Name three character' st ics that differentiate programmed instruction 
material from non -prog rammed material. 

Define task analysis in reference to instructional design. 

8 Briefly define what is meant by entry behavior (give examiples). 

9- B'iefly define: 

a. Baboon frame-- 

b. Discrimination f:dir.e-- 

c. Confirmation frame-- 

d. Sub-terminal frame-- 

10. A wel 1 written behavioral objective contains: 
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11. List and briefly define Gagne's eight levels of learning, 

12. Briefly define: 

a. Adjunct programniing'- 

b. Extrinsic programirnng-- 

c. Intrinsic programming-- 

d. Linear programming— 

13. List at least two prog’^amming strategies and reasons for selecting each. 

14. List and define at. least four types of test items. 

15. List five characteristics of a well -written test item. 

16. Briefly differentiate between formative and summative evaluation. 

17. List the differences between using the Systems Approach in developing 
materials for programmed instruction and materials for any other medium. 



r * 
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APPENDIX C 



537 PROJECTION EVALUATION SHEET STUDENT NAME 

CMI # 

evaluTHW 

Topic Area 

Expert 

(10) 1. Task Analysis (5) 

Reasonable sequence (5) 

(10) 2. Behavioral Objectives (5) 

Well written, compared with TA (5) 

a. Comparison with conventional curricula 

b. Outline of contents of program 

c. Show limits of the program's objectives and 
areas not being developed by the program 

(5) 3. Description of Entry Behaviors (3) 

How derived, expressed as entry behavior and 
skills (2) 

(5) 4. General Description of Target Population (5) 

Not entry behavior 

a. Minimum grade on standardized ability or 
aptitude tests 

(10) 5. Program Preparation (Description of preparation 
process) (10) 

Select strategy, use TA, entry behaviors, etc. 

a. Expert Opinion 

b. Revision based on one-on-one 
(10) 6. Evaluation Plan (Preparation for evaluation, 

data collection and development of instruments) 
and report of data (error rate of program and 
testing) (10) 

(10) 7. Revision Suggestions or Implications (10) 

Based on Report of Data in #6 

(5) 8. Miscellaneous (General overall impressions) (10) 

a. Indications of student attitude 

b. Practicality of program, e.g., supplemental 
materials, reuseabi 1 ity , maintenance costs, 
etc. 

c. Administration, e.g. , conditions necessary for 
success, procedures forintroducing students;:to 
the program, etc. 
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